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ABSTRACT 



Detroit's Peer Teachers as Mlrrore and Monitors Project Is a 
National Institute of Education sponsored research project Intended 
to validate cost effective methods for Increasing Academic Learning 
Time (ALT) for stu^Jents In Grades one through four* 

A major problem In this research effort has been the design of 
valid and reliable measures of the components of ALT* One very 
Important component of ALT Is student and teacher use of time* 

This presentation will address some of the Issues Involved In 
measuring student time-on-task, and teacher's use of time with 
Ftudents* Observation methods which produce valid and reliable 
data without Interrupting classes or threatening teachers (too 
much) will be discussed* Samples of observation Instruments 
developed by the project will be included in the presentation along 
with issues related to training observers and the establishment of 
definitions of behaviors to be observed so as to produce high 
inter-rater reliability* 
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ME\SURING ACADEMIC LEAMING TIME: SOME INSIGHTS 
THROUGH THE LOOKING GLASS 



Introduction 

In the eAvly 1980*s» educators began to focus attention on the use of 
time In claasrooms as a variable which might explain students* lack of 
achievements This focus was the result of considerable research concerning 
the relationship between engagement rates and achievement (e,g, , Fredrick, 
Walberg and Rasher, 1979; Stalllngs, 1976 and 1980; Good and Beckenaan, 1978; 
Davidson and Holley, 1970; Anderson and Scott, 1978; Fredrick, 1977; 
Rosenshlne, 1976; Fredrick and WaXberg, 1980), In 1981, the National 
Institute of Education Issued a request for proposal for research contracts to 
Investigate cost effective niethods for Increasing student tlme-on^task for 
students In grades 1-4 who attended Title I eligible schools* The monies for 
these research contracts were allocated from Follow Through funds which were 
set aside to fund research In promising practices for raising student 
achievement at these gr&de levels » 

The Detroit Public Schools was one of four districts across the United 
States to be awarded such a contract. The project vhlch resulted tram this 
award was entitled, *'Peer Teachers as Mirrors and Monitors*' and attempts to 
Increase the mathematics and reading achievement of students In grades one 
through four by Increasing the amount of time students ^re engaged In learning 
tasks In vhlch they experience a low error rate and which are directly related 
to the outcome measures* The strategy for accomplishing this purpose Includes 
a periodic syster of feedback In vhlch teachers are given data by their peers 
which document their students* engaged-ln-learnlng rates as well as their own 
classroom management procedures (Mirrors and Monitors) plus four different 

1 
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types of teacher training* Including: Knowledge of Theories and Practices* 
Modellng/Demonstratlon/Sharlng* practice In Simulated Condltlona* and 
Coaching/Recycling* 

The Mirrors and Monitors data required the design and development of 
Instruments which would provide valid and reliable data on student on-taak/ 
off-task behaviors and on teacher behaviors which could be linked to student 
behaviors In the classroom* Since teachers Vi^uld b^ collecting data In peers* 
classrooms* the Instrumentation and collection procedures had to be simple 
enough so that all teachers Involved In the project would feel comfortable 
using them and unobtrusive so as not to Interrupt the normal classroom 
routine* The data collected were to be shared between the observer and the 
observed peer teacher which meant that the Interpretation of the data had to 
be slisple and straightforward* 

Once the Instruments were developed* training procedurea for teachers tind 
other data collectors had to be designed and tested* 

This paper will describe the Instrumenta which were developed for 
collecting student on-task/off-taalc data and teacher /classroom level data* the 
validation procedures uaed to test theae Instrumenta* the training procedures 
uaed with data collectors* and some of the first year results gathered with 
the Instruments described* 
Student Level Data 

Data on atudenta* on-taak/of f-tasic statua la collected for every student 
In the claasroom every two itlnutea during the class period* This provides a 
aample of behavlora for each atudent throughout the period In the manner of a 
^'snapshot" of the classroom taken every two nlnutea* When the sample of 
behavlora Is averaged for each student* It gives an approximation of the 
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proportion of the cisss period during vhlch the student vss either on-task or 
off-task* Individual observations for each student may also be compared to 
data collected on the teacher^s behaviors In order to diagnose reasons why 
students may be off-task or on-task during psrtlcular parts of the class 
period* 

In order to collect data on every student during every two minute 
Interval* a seating chart format Is used (see Figure 1)* Prior to the 
observation* a seating chart which gives the name snd location of esch student 
In the classroom Is prepared* Each student^s name appears In a rectangle 
which Is Isrge enough to accomodate five rows of five marks within the block* 
The marks represent the students* status as follows: a plus sign (+) for 
on-task* a minus sign (-) for off-task^ and a zero (0) for don't know* For 
students In the lower grades where regrouping Is a freque^it occurance^ a list 
of students* names Is substituted for the seating chart* In this case* the 
codes ar& recorded adjacent to each student^s name* 

On-task behavior Is defined as* "students participating In the Intended 
lesson which Is related to either reading or mathematics*" Examples of 
on-task behaviors Include: participating In guided lessons* responding 
orally* engaged in a written assignment* engaged In a discussion that Is 
related to the lesson* and taking a test or quiz* Off-task behavior is 
defined as^ ^'behavior not related to the lesson or a lack of involvement on 
the part of students*" Examples of off-task behaviors include: engaged in a 
social interaction^ uninvolved in the lesson^ exhibiting dlaruptive behavior* 
waiting for help^ being disciplined* sharpening pencils^ daydreaming^ and 
roving about the room* Observers are instructed to attempt to code students 
either on- or off-task if at all possible* The zero (0) code is used only 
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FIGURE 1 

DETROIT PUBLIC SCHOOLS 
PEER TEACHERS AS MIRRORS AND MONITORS 
Student Time-On-Task observation Forn^ 

Teacher Time : to 

School Grade: Subject 

Room t Date 

Observer . 



SEATING CHART I 
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when the observer Is unable to observe the student for some reason* 

Coding student behavior Involves rapid decisions on the part of the 
observer* A sweep of the classroom Is made at the beginning of each tvo 
minute observation period* Each sweep follows the same path In order to 
assure that students are observed In approximately the same segment of the 
time Interval* As the observer glances at each student* he/she observes and 
makes a decision about the student^s on-task/of f-task status and records the 
appropriate mark on the observation form* No attempt Is made to make 
cumulative judgements concerning the students* behavior during the entire two 
minute Interval* Since each of the 25 observations represents only four 
percent of the total observation* e^^Tors resulting from this procedure are 
minimal* 

Classroom Level Data 

Teacher/classroom level data are collected simultaneously with student 
level data In order to provide feedback to teachers on their behavior and on 
the classroom configuration and activities during the observation period* One 
of the Important goals of the Detroit project Is to Increase teachers* use of 
Instructlonally appropriate behaviors while decreasing those teacher behaviors 
which cause students to move off-cask or which* based upon current research* 
probably fall to Increase student learning* The data collection Instruments 
which mi*e developed record specific teacher behavlorj& and document activities 
In which members of the class were Involved during each of the two minute 
observation periods (see Figure 2)« 

The activities listed on the classroom/teacher observation form are 
divided Into four categories: non-lnteractlve Instruction* Interactive 
Instruction* off-task* and organization* Non-Inter active Instruction Includes 
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peen teachers as Htnitoits and monitors 

Observation Form 

DIRECTIONS: Durinf each 1 minute observation, record the ntmber of students involved in each of the activities listed below. 
Circle the ntmber of students involved in activity in which t}ie teacher is directly participating. 



School : 



Teacher: 



Start tl»«: 



t m 



Observation f 



End tiaie: 

3 4 5 6 



Grade: 



Roon: 



Subject: 



Observer: 



Date: 



Nuaiber of students: 

Nynber of 

10 11 12 13 14 IS 16 17 16 19 20 21 22 23 24 25 Circles 



Silent Readliu 






















































Se«t Work 






















































Or«l IteadiOi 






















































insirtictionai 
Explanation 
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SUMMARY INFORMATION 



Non* Interactive Instruction 
Interactive Instruction 
Off-Task 
Org an i I* t ion 



total nunber 
teacher circles 
in each area 



percen of 
observations 
in each area^ 



target 

per cents for 
each aroa 

351 or less 
sot or nore 
St or less 
ISt or less 



^Divide the toUl nuaiber 
of teacher circles by the 
number of observations «nd 
ntiltlply your Answer by 100 
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silent reading and seat work* Interactive Instruction Includes oral reading* 
Instructional explanation* giving directions* dlscuss/revlew assignments* and 
practice/drill* Off-task behaviors Include the number of students off-task 
as defined In the student form* Organization Includes management* distribute/ 
collect materials* and transitions* Specific definitions were developed for 
each of these behaviors (see Figure 3) based upon Input from the project 
teachers * 

As a guide to project teachers In the Interpretation of the results 
obtained from using this observation Instrument* target percen'cs for each of 
the four areas addressed on the form are Included* These target percents are 
based upon the work of Stalllngs (1980)* She found that the most effective 
teachers used Interactive Instruction at least 50 percent of the time* used 
non-lnteractlve Instruction* at most* 35 percent of the tlme» used 15 percent 
or less of the time for organizational activities* and used 5 percent or less 
of the Instructional time In off^task behaviors* 

Each observation period consists of a tvo minute Interval during which 
each student Is observed and his/her on-task status recorded on the student 
observation form after which the number of students Involved In each of the 
activities listed on the i:eacher/classroom observation form Is recorded In the 
appropriate location* Finally, the activity In which the teacher Is directly 
Involved Is Indicated by circling the n*amber of students Involved In that 
activity* The student and teacher/classroom observation forms are completed 
during each tvo minute Interval* During a 50 minute class period* 25 
observations are recorded for each atudent and 25 sets of numbera are recorded 
on the teacher/classroom observation form* 
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P»t TfACaCftS AS HllsaES AKD MaKlTDftS 



ACTIVITY 



DESCliyTlQM 



Sllettt R««dlns 



5««t Work 



ln«triic tlon&l 



D1«cii«»/R«t1«v 



Fr«etle* /Drill 



Nan* f*««ftt 



Dl«trlbot«/colX«et 



Stiid«Qt« «r« r«<(dl&f «ll«ntly to th«ft««lv«« 
«• « group flctlvlty or «re vorlclng on 
IfidlTldufll «««lf&««tit«* Mo vrltl&f* 

On* or ftor« *ttid«cr« l«/«r« vrltlsg p«p«r«t 
dolttf cofeputatloo, or Involvfld lo mny otb<r 
«ll«ot vrltt«tt vork r«Iat*d to tb* l«a«oti* 

On« or «or« «tiid«ot« l«/ar« r«adlof a 
«*ctloB fro« « P^«T «loiid or roadlog « book 
for tb« elaoo ro roadltif group to boor* 
Orol rvfrdlng 1« uoually not dono in unloont 
fonorollyt ntudotitn toko tumo rondlng 
ooquootlnl nnctlonn froft m book* Tbo 
toocbor or tbo otudont(o) v'on <^loo rood 
nloud vbllo tbo root of tbo clnoo follow 
nlong Id tbolr ovn toxto* 

An odult lo Inforfelng oofto grouplftf of 
otudOBto nbout n oubjoct. Aendofelc 
dlocuooloo or olov-pocod quootlon/onover 
oooolon tnknn pineo rofordlng Xoeturo 
fentorlolf ooolgnftontot or probXofto. 

An ttdult in oxplolAlnf on oetlTlty* tbo 
procoduroo to bo foXlovod* tbo oftount of 
vork to bo flolobod* or rovnrdn for 
eoftplo..lsf tbo ooolgnfeoot* Tbo dlocuooloo 
lo oot foeuolftf oo i;bo nendo«le eontont* 
but on tbo Inforftotloo tbnt otodonto ftood 
to enrry out tbo ooolgMoot <or dloeooolng 
grndoo)* 

Ooo or «oro otodonto lo/nro roeolvlng 
loforsotlon or foodbo^k on vork tboy bnvo 
coftpXotodt or nro bolog oToXuntod on cbolr 
vork propnrotory to eootln*tlnf tbo 
noolf n&ont * 

Ooo or «oro otndonto lo/nro TorboXly 
lOToXvod In rolnforcloft ropotltlvo* or 
roto vork* Tblo netlvlty «aot bo 
dlff orontlntod fro« ooot vork* Stodonto 
vrltlof Torbnl motorlnlt oo In dletnclont 
oro nXoo eodo^ Frnetlcft/DrllX* 

Otto or ftoro otndonto or toocbor ottd otudonto 
oro Intoroctlnf «bout vork or oubJocto otbor 
tboo' clooo-roXotod «otorloXt or «todonto oro 
not IttToZvod ift 007 octlvltyt oro orrlvlng 
or XooTlOft or aovlog 4bout tbo roo«* <5oo 
Hot of off^t^ok boh^fvloro) 

Toklttt otT'indo^^Ot SoklOf/rocolYlnt 
oMOOttcikMootcot rogrooplftft for«lttf llnoot 
dlocirXlttot colXoctlOf «o«ityt otc* 

T^iocbor itftd/or otodonto nro IttToXvod lo 
^ooolttf out MF*TOt rottlag ovny votorloXo* 
1»ropnrlof to Xooto* or pro^nrlng or 
cfaocklOf feotorlnXo* 

CbnOflng fro« o&o nctlvlty to n&otboV* 
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Observers summarize the observation data recorded on the teacher/ 
classroom level observation form by counting the number of observation periods 
the teacher was Involved In each of the activity areas* This number Is 
converted to a percent by dividing It by the number of observation periods In 
vhlch data vere recorded and multiplying the result by 100* The resulting 
percent Is comparer to the target percent for each of the four areas In order 
to give the observed teacher an Idea of how she/he compares to the research 
based percents and In vhlch areas she/he may need to change behavior patterns* 
Additional Information about the effect of various types of behavior on the 
part of the teacher can be determined by comparing the two observation forms* 
For each observation period the behaviors observed and the on-task/of f-task 
status of each student can be compared In order to give the teacher additional 
Information about what specific activities students In the classroom were 
involved In and what types of activities nay have caused them to stray 
off-task* Grouping patterns are also apparent from the data on the foxms 
Including the numbers of students with whom the teacher was working during any 
given observation period* All of these data provide Information and guidance 
for changing behavior and/or reinforcement for behaviors which result In high 
on*task levels for students* 
Validation Procedures 

All observation forms developed by this project were field tested and 
validated over a one-year period prior to project classroom use* The 
validation procedure Involved criticism and revision of the forms by project 
staff » project teachers and administrators » and outside consultants 
knowledgable In this type of data collection* Field tests of each of the 
Instmments were conducted In Detroit schools not Involved In the project at 
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all elementary grade levels and also with high school students* The results 
of the field tests Indicated that the forms produced data which appeared to 
represent the actual classroom situations obaerved* Field teats were 
conducted by pairs of observers and produced high inter -observer reliabili- 
ties* (No atatistical inter-observer reliabilities were computed since only 
two observers were involved in the initial field tests and insufficient 
numbers of observations were available to produce meaningful statistical 
information* ) 
Observer Training 

Observer training requires two phases* First* observers must le&rn the 
definitions of the behaviors to be observed and become familiar with the 
location of each of the behaviors on the classroom observation form and with 
the definitions of on^task and off'^task behaviors for students* Second* they 
must be provided with guided practice and independent practice in using the 
observation forms to code behaviors of students and teachers in a closely 
supervised situaticn* 

The first phase of the training is best accomplished through presentation 
of the concepts involved in on-task/of f-taak behavior data collection and a 
detailed walk-through of the definitiona of each of the behaviors with ample 
examples presented to the group of persons to be trained* Observers ahould 
then be informftd that they muat memorize these definitiona and be able to 
discriminate among all the behaviors liated cn the fora* 

The guided practice phase of the training should begin with the use of 
video tapes of actual claasroom situations which may be stopped and restarted 
at any deaired point* Theae aituationa are ufted to present the group with 
classroom situations which they might encounter during the obaervation period* 
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Group consensus Is reached about hov each of the observed situations should 
be coded on the teacher /classroom observation form* Following the use of 
video tapes* the group of observer trainees should be divided Into two 
approximately equal groups* One group will role play a classroom situation In 
which one of them Is teaching a lesson and the remaining Individuals act as 
students, Whllft the role play takes place* the other half of the obstrver 
trainees codes the teacher and students « After about 20 minutes , the 
observers are debriefed and their coding forms are compared and discussed by 
the entire group* Then the two groups switch roles and the process Is 
repeated* 

After Initial training* the observers should be sent to observe actual 
classrooms* These observations should be conducted In pa:<,rs so that the 
Inter^observer reliability for the observers can be checked* Following each 
observation* the pair of observers should share their results and discuss with 
the trainers any discrepancies they observe* At leaet three practice codings 
should be conducted before actual data collection Is attempted by newly 
trained observers* 

Most Individuals can learn to use the forms and collect valid and 
reliable data In two days* The project has trained over twenty observers and 
experienced a high degree of success* The Initial apprehension expressed by 
most observers prior to training Is that the two minute observation Interval 
might be too short to collect the required data on all students and to record 
a snapshot observation of the classroom configuration* Following training and 
practice* nearly all found that they had more than enough time to collect and 
record the required data* 
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Sample Pita 

To illustrate results obtained from using these forms » data on students 
and teachers ^t three schools with students from grades 1-4 are presented* 
Tliese data were collected during the fall of 1982, 

TABLE 1 



Means and Standard Deviations of Student 
On-Task Rates at Three Schools 
Fall, 1982 Data 



Participants 


Grades 


M 


On-Task 
Mean 


Percent 
S.D. 


School A 












Teacher 


One 


1 


26 


82% 


8% 


Teacher 


Two 


1 fir 2 


24 


61 


7 


Teacher 


Three 


3 


18 


85 


12 


Teacher 


Four 


2 


25 


84 


3 


Teacher 


Five 


3 6 4 


16 


68 


11 


Teacher 


Six 


3 


26 


91 


6 


Teacher 


Seven 


2 


14 


76 


11 


Teacher 


Eight 


1 


24 


88 


4 


Teacher 


Eleven 


4 


46 


95 


6 


Teacher 


Twelve 


2 


13 


67 


3 


School B 












Teacher 


Fourteen 


2 


24 


81 


18 


Teacher 


Fifteen 


1 


27 


62 


11 


Teacher 


Sixteen 


1 


24 


72 


4 


School C 












Teacher 


Seventeen 


3 


33 


79 


9 


Teacher 


Eighteen 


1 


27 


61 


5 


Teacher 


Nineteen 


1 


29 


56 


12 


Teacher 


Twenty 


4 


29 


85 


. 4 


Teacher 


Twenty-One 


2 


32 


81 


4 


Teacher 


Twenty-Two 


2 


31 


95 


4 


Teacher 


Twenty-Three 


2 


33 


56 


8 



Table 1 above presents m^^ans and standard deviations of students on-task 
rates for twenty teachers at three schools. The on-task percent for students 
in these twenty classrooms varied from a low of 56% to a high of 95%, 
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Table 2 presents the mean percent of teacher/classroom level behaviors In 
each of the four categories measured by the teacher/classroom level form 
during the same observations presented for students In Table 1* 

TABLE 2 

Mean Percent of Teacher/Classroom Level Behaviors 
For Teachers at Three Schools 
Fall, 1982 Data 



Participants 


Interactive 


Non-Int er ac ti ve 


Off-Task 


Organization 














Teacher 


One 


7Z 


88S; 


0% 


5Z 


Teacher 


Two 


9 


68 


4 


19 


Teacher 


Three 


9 


81 


0 


9 


Teacher 


Four 


27 


66 


0 


7 


Teacher 


Five 


25 


49 


1 


25 


Teacher 


Six 


10 


83 


2 




Teacher 


Seven 


18 


60 


3 




Teacher 


Eight 


6 


88 


0 


6 


Teacher 


Eleven 


4 


92 


0 


4 


Teacher 


Twelve 


6 


70 


0 


24 


School B 












Teacher 


Thirteen 


6 


62 


11 


21 


Teacher 


Fourteen 


14 


79 


1 


6 


Teacher 


Fifteen 


6 


70 


2 


22 


Teacher 


Sixteen 


10 


70 


0 


20 


School C 












Teacher 


Seventeen 


8 


68 


3 


20 


Teacher 


Eighteen 


31 


40 


2 


27 


Teacher 


Nineteen 


25 


50 


9 


16 


Teacher 


Twenty 


28 


51 


0 


21 


Teacher 


Twenty-Ofie 


12 


78 


1 


2 


Teacher 


Twency-Two 


36 


69 


14 


0 


Teacher 


Twenty-Three 


11 


68 


6 


15 
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Summary 

Experience gained from one and one-half years of data collection In the 
area of tlme^on^task In conjunction with Detroit's "Peer Teachers as Mirrors 
and Monitors** has given the author a more realistic perspective on the 
difficulties ^nd problems associated with this endeavour* This paper has 
attempted to present a valid and reliable method for collecting data on 
students and on teacher /classroom level behaviors which can be used to collect 
these data with a minimum of interruption to the classroom routine and vhlch 
teachers and other data collectors can use with a minlmam of training and 
difficulty* Data were presented vhlch were collected using these Instruments 
and copies of the instruments were included* 

The results of the data collected using these Instruments are useful to 
teachers who have been observed* They give feedback to teachers relative to 
their behaviors and teaching practices and information on every student in 
their classroom over an entire class period* The behaviors observed are a 
subset of all possible behaviors which could be observed in classrooms* It is 
recommended that teachers who are to be observed have input into which 
behaviors are to be observed and that opportunities for help be offered to 
teachers who fall to meet the targets in any given behavioral area* It is 
also suggested that student data be used by teachers to target instruction and 
to examine the appropriateness of content and difficulty level of tasks 
assigned tc students who show high rates of off^task behavior* 
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